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Abstract 

Background: Some level of monitoring is usually required during a clinical trial to protect the rights and safety of 
trial participants and to safeguard the quality and reliability of trial results. Although there is increasing support for 
the use of risk-proportionate approaches to achieve these aims, the variety of methods and lack of an empirical 
evidence base can present challenges for clinical trial practitioners. 

Methods: This paper describes the monitoring methods and procedures that are utilised by a non-commercial 
clinical trials unit which coordinates a range of clinical trials across a variety of clinical areas with different 
associated risks. 

Results: Monitoring activities and approaches should be selected to be proportionate to the risks identified within 
a trial. A risk-proportionate approach to monitoring is described giving details of methods that may be considered 
by clinical trial practitioners during the development of a trial monitoring plan. An example risk assessment and 
corresponding monitoring plan for a low risk (type A in the Medicines and Healthcare Products Regulatory Agency 
(MHRA) classification system) pediatric trial is provided for illustration. 

Conclusion: We present ideas for developing a monitoring plan for a clinical trial of an investigational medicinal 
product based on our experience. Alternative approaches may be relevant or preferable in other settings based on 
inherent risk. 

Keywords: Monitoring, Central monitoring, On-site monitoring, Risk proportionate. Quality assurance 



Background 

Trial monitoring is defined by the International Confer- 
ence on Harmonisation of Good Clinical Practice (ICH 
GCP) as 'the act of overseeing the progress of a clinical 
trial and of ensuring that it is conducted, recorded and re- 
ported in accordance with the protocol, Standard Operat- 
ing Procedures (SOPs), Good Clinical Practice (GCP), and 
the applicable regulatory requirement(s)' [1]. ICH GCP 
also states that 'the purposes of trial monitoring are to ver- 
ify that: (a) the rights and well-being of human subjects 
are protected, (b) the reported trial data are accurate, 
complete, and verifiable from source documents, (c) the 
conduct of the trial is in compliance with the currently 



* Correspondence: cat 1(5) I iv.ac.uk 

^Clinical Trials Research Centre, Department of Biostatistics, and North West 
Hub for Trials Methodology Research, University of Liverpool, 1st floor 
Duncan Building, Daulby Street, Liverpool L69 3GA, UK 
Full list of author information is available at the end of the article 

(3 BioMed Central 



approved protocol/amendment(s), with GCP and with the 
applicable regulatory requirement(s)' [1]. 

The ICH GCP guidance is not specific about which 
methods should be used but suggests that 'the extent and 
nature of monitoring should be based on considerations 
such as the objective, purpose, design, complexity, blind- 
ing, size and endpoints of the trial' [1], The guidance 
highlights a general need for on-site monitoring during 
different phases of the trial, but recognizes that 'in ex- 
ceptional circumstances the sponsor may determine 
that central monitoring in conjunction with procedures 
such as investigators' training and meetings, and exten- 
sive written guidance can assure appropriate conduct of 
the trial in accordance with GCP' [1]. However, this has 
been criticized in the literature, with concerns raised 
that inefficient methods of monitoring are being used 
unnecessarily in some trials due to misinterpretation of 
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the guidance [2] and a misconception that on-site moni- 
toring is a legal requirement This has in part led to recent 
initiatives on risk-adapted approaches to monitoring from 
the Clinical Trials Transformation Initiative (CTTI) [3], 
Department of Health [4], Food and Drug Administration 
(FDA) [5], and the European Medicines Agency (EM A) 
[6]. These are substantial developments, both for commer- 
cial and non-commercial clinical trials, and will provide 
the potential to reduce costs and increase efficiency. How- 
ever, there is a lack of empirical evidence to determine 
which practices best achieve the goals of monitoring 
stated in ICH GCP [3] and, consequently, heterogeneity in 
methods of monitoring [7]. Although some empirical evi- 
dence is emerging in the literature [8-11], there are very 
few published examples of monitoring methods that are 
used in practice, which limits the potential for the sharing 
of practical experience and expertise to assist clinical trial 
practitioners developing monitoring procedures to use in 
practice. The aim of this paper is to describe the risk- 
proportionate approach to monitoring, with details of cen- 
tral monitoring as well as on-site monitoring methods, 
currently undertaken at the Clinical Trials Research 
Centre (CTRC), University of Liverpool, to improve ac- 
cess to, and encourage sharing of practical methods. 

Methods/Design 

Risk-proportionate approach to monitoring adopted at 
the Clinical Trials Research Centre (CTRC) 

The CTRC, based at the University of Liverpool, UK, 
was established in 2007 and gained full registration sta- 
tus as a United Kingdom Clinical Research Collaboration 
(UKCRC) clinical trials unit in 2009. The UKCRC has a 
network of 45 registered Clinical Trials Units (CTUs) 
which have provided evidence to an international panel 
of experts of their capability to coordinate multi-centre 
clinical trials (having overall responsibility for the design, 
development, recruitment, data management, publicity 
and analysis of a portfolio of trials), and of robust sys- 
tems to ensure the conduct and delivery of clinical trials 
to the highest quality standards. The portfolio of trials 
that are designed, coordinated and analyzed at the 
CTRC include pediatric, obstetrics and gynecology, 
neurology, infection, and dental trials involving investi- 
gational medicinal products, devices, and surgical tech- 
niques. The trials vary considerably in target sample 
size, the number of recruiting sites, length of follow up, 
and experience of research staff at recruiting sites, but 
all trials would adopt a risk-proportionate approach to 
monitoring as described in this manuscript. 

In terms of the potential risk associated with the In- 
vestigational Medicinal Product (IMP), the majority of 
trials coordinated by the CTRC would be categorized as 
'Type A (no higher than the risk of standard medical 
care) or 'Type B' (somewhat higher than the risk of 



standard medical care) according to the MRC/DH/MHRA 
Joint Project guidance on Risk-adapted Approaches to the 
Management of Clinical Trials of Investigational Medi- 
cinal Products [4]. Each trial has three oversight commit- 
tees: the Trial Management Group (TMG) concerned 
with day to day running of the trial, the Independent Data 
and Safety Monitoring Committee (ID SMC) to view trial 
arm comparisons of safety and effectiveness, and the Trial 
Steering Committee (TSC) that considers the recommen- 
dations of the IDSMC and makes the ultimate decision 
for the continuation of the trial. A separate charter is de- 
veloped to describe the membership, planned frequency 
of meetings, roles and responsibilities, and interactions 
between these committees, which would be referenced 
within the monitoring plan. 

The stages involved in the risk proportionate approach 
to monitoring adopted by the CTRC are briefly summa- 
rized in Figure 1. For each trial, a structured risk assess- 
ment is undertaken to identify potential patient, study or 
organizational hazards (see example risk assessment in 
Additional file 1). 

This requires input from the multi-disciplinary trial 
team, including statistics, data management, trial manage- 
ment and clinical input. The CTRC Operational Team 
reviews all CTRC Risk Assessments and final approval 
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of each completed risk assessment is obtained from the 
CTRC Director, Sponsor and Chief Investigator. The 
risk assessment form contains both generic and specific 
hazards and for each hazard it should be documented 
whether that hazard is, or is not, applicable to the trial 
Any additional trial specific hazards should be added to 
the risk assessment. The total, mean, and overall per- 
centage risk scores are calculated for the trial (using for- 
mulae described in Additional file 1) to provide an 
overall guide and trial risk classification. However, indi- 
vidual risk scores should also be examined closely (using 
the risk management matrix and key in Additional 
file 1) to ensure that appropriate strategies are in 
place for monitoring hazards with particularly high risk 
scores. The allocation of scores can be subjective and 
alternative approaches to risk assessment, which do 
not require the calculation of a numerical score, may 
be appropriate. 

The extent and nature of the monitoring required is 
determined by the risk mitigating activities to be under- 
taken and specified within the monitoring plan (example 
in Additional file 2) with specification of escalation of 
monitoring activities. The aim of the monitoring plan is 
to describe how problems that could affect the rights 
and safety of participants or the reliability of study re- 
sults can be prevented, or detected in a timely fashion 
through appropriate monitoring and subsequent cor- 
rective action. Regular trial monitoring reports are pro- 
duced and used to determine when action such as 
additional site training or on-site visits are required as 
specified in the monitoring plan. For 'Type A classified 
trials the monitoring plan would usually describe 
mostly centralised procedures with triggered on-site 
visits and would highlight examples of triggers for fur- 
ther action. The risk assessment and monitoring plan 
may be revised in light of protocol amendments or 
monitoring reports. 

Developing the monitoring plan 

The CTRC monitoring plan is a document which de- 
scribes all monitoring activities for a particular trial and 
details planned central monitoring, on-site monitoring, 
oversight committees, the roles and responsibilities for 
undertaking activities, and relevant timelines. The plan 
would also be developed with appropriate consideration 
for the size of the trial as some monitoring methods 
may be restricted for small trials. Examples of monitoring 
activities that would be included within the monitoring 
plan are described in the following sections. An illustrative 
example is provided in Additional file 2 which relates to a 
pediatric CTRC 'Type A Trial of Optimal TheRapy for 
Pseudomonas EraDicatiOn in Cystic Fibrosis (TORPEDO- 
CF). The summary protocol and corresponding risk assess- 
ment are also provided (Additional files 1 and 3). 



Responsibilities 

The Sponsor has ultimate responsibility for the quality 
and integrity of the trial data [1] but may delegate activ- 
ities whilst maintaining appropriate oversight. The multi- 
disciplinary nature of the TMG and the multilevel nature 
of monitoring approaches demand the need for individual 
responsibilities and expectations to be clearly defined 
within the monitoring plan. The CTRC monitoring plan 
development team includes the Trial Coordinator (TC), 
Trial Statistician (TS), Trial Monitor (TM) (if relevant) 
and Chief Investigator (CI) of the trial. This development 
team structure is regarded as the minimum requirement 
with input sought from additional expert members, such 
as a pharmacist, where specific hazards have been identi- 
fied. The TC takes overall responsibility for coordinating 
the production of the monitoring report. The CTRC 
Senior Management Team (SMT), which consists of the 
CTRC Director, Head of Statistics, Head of Trial man- 
agement. Head of Information Systems, Quality Assur- 
ance Manager, and Head of Data Management, review 
and sign off the monitoring plan for each trial before 
implementation. 

Results and discussion 

Trial monitoring approaches 

There are essentially two approaches for monitoring 
clinical trials: central monitoring, and on-site monitor- 
ing, with the monitoring plan providing a summary of 
the practical procedures and reporting summaries re- 
quired to undertake these activities. 

Central monitoring 

Central monitoring involves centralized procedures for 
the quality control of trial data. Responsibility for each 
section of the monitoring report should be specified 
within the monitoring plan. There are numerous central 
monitoring procedures that may be considered for an in- 
dividual trial. Some approaches involve using statistical 
methods (central statistical monitoring) to explore pat- 
terns in the accumulating data, some involve monitoring 
results of automated validation checks that may be built 
into the data management facility, some may involve the 
central review of forms submitted to the CTRC, and 
some may involve a comparison of performance across 
participating sites. Defining an acceptability threshold', 
such that the crossing of this threshold would trigger ac- 
tion, may also be helpful to guide decision making for 
some monitoring activities (for example see Section 
missing primary outcome data ) These central monitor- 
ing methods are used to assure the quality and reliability 
of trial data, protect the safety of trial participants, de- 
tect trial conduct problems at particular sites, and can 
be useful for detecting fraud [12]. Listed below and in 
Additional file 2 are examples of central monitoring 



Tudur Smith et at. Trials 2014, 15:127 
http://www.trialsjournal.conn/content/1 5/1/1 27 



Page 4 of 10 



procedures that would usually be considered for CTRC 
trials, although it is recognized that this is not an ex- 
haustive list. 

Review of consent forms 

Documenting that the consent process has taken place 
and that trial participants have been fully informed 
about the trial is an essential process to safeguard the 
rights of trial participants. In general, copies of com- 
pleted consent forms are faxed to the CTRC within 
seven days of completion by participating sites. Each 
form is then checked using a consent form checklist to 
ensure that each form has been fully and accurately 
completed and returned from sites within a reasonable 
time frame. Any issues identified through this routine re- 
view of consent forms is summarized in the trial monitor- 
ing report to enable the overall site specific issues to be 
reviewed. As patient signatures are included on completed 
consent forms, all faxed copies are kept securely in a 
locked cabinet, separate from other trial documents. Pa- 
tient names are not routinely recorded or linked in any 
way with further data which may be collected during the 
trial. Furthermore, the patient consent form explicitly re- 
quests permission for the completed consent form to be 
passed to the CTRC for the administration of the study. 

Checks across individual sites for problems 

In multi- centre trials, data from one site can be com- 
pared with data for the whole trial to compare perform- 
ance and identify patterns which may indicate that 
action is required. The key data items to be compared 
across sites should be identified after consideration of 
the risk assessment and listed in the monitoring plan. 
Methods for statistical monitoring such as exploring 
digit preference, rounding, or unusual features of a fre- 
quency distribution (such as outliers, inliers, or atypical 
degrees of skewness or kurtosis) which have been de- 
scribed [2] for the detection of data fabrication or data 
falsification, may be useful when considering the com- 
parison of data across sites. Graphical approaches such 
as site-specific box and whisker plots of key continuous 
variables, or graphical approaches described in later sec- 
tions, can help with interpretation of site comparisons 
and facilitate decision making. However, it should be noted 
that differences in participant characteristics across sites 
may explain apparent differences for certain types of vari- 
ables. Furthermore, the application of some methods for 
statistical monitoring are limited in small trials and the 
issue of multiple testing needs to be considered carefully if 
significance tests are to be performed. 

Recruitment and retention 

Recruitment of participants below the expected or pre- 
dicted recruitment rates can jeopardise the power of the 



study, impose time delays and financial penalties on the 
trial, and can lead to premature trial closure. Frequent 
monitoring of recruitment data will help to identif)^ 
whether the target sample size can be achieved within 
the expected timeframe, and any appropriate corrective 
action that may be required. 

The frequency of non-eligible patients and eligible pa- 
tients who do not provide consent (with a summary of 
the reasons) compared across sites can give valuable in- 
formation about the recruitment process, including re- 
strictive eligibility criteria or issues in its application or 
interpretation. Comparison of site specific retention 
rates (percentage of patients at a site that have with- 
drawn) can highlight sites where there may be issues 
with following the correct consent process. 

The benefits of monitoring recruitment by site include 
identification of sites which are recruiting well to deter- 
mine successful recruitment initiatives relevant to other 
sites, additional site-specific support can be provided for 
poor recruiting sites or poor recruiting sites can be 
closed and resources redistributed appropriately. Early 
identification is essential to allow adequate time for 
planning additional trial-specific and site specific re- 
sources. An example graph showing overall recruitment 
and site openings is presented in Figure 2. Similar site 
specific graphs or graphs showing site specific recruit- 
ment rates per unit of time can also provide valuable 
information. 

Randomization system 

Adequately concealed randomization systems eliminate 
selection bias and produce groups of participants that 
are balanced with respect to known and unknown prog- 
nostic factors. All randomized trials should be monitored 
to ensure that the randomization system is working cor- 
rectly and that participants are being randomized prop- 
erly. These checks would be undertaken by the TS and 
would usually coincide with the preparation of data 
monitoring reports presented to the IDSMC. However, 
a minimum check should be scheduled during early 
stages of recruitment to promptly identify problems and 
apply corrective action. The risk assessment may indi- 
cate that more frequent monitoring is required if, for 
example, randomization envelopes or a web-based mini- 
misation procedure are being utilised. The appropriate 
checks to undertake will depend on the method of 
randomization but would include a summary of baseline 
characteristics and number randomized between treat- 
ment groups (usually presented in the IDSMC report), 
as an imbalance could indicate a problem with the 
randomization system. If stratification has been used in 
the randomization process the number randomized in 
each treatment group across stratification variables 
should be summarized. Depending on the implications 
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Figure 2 Overall recruitment graph. 



for unblinding, details of such randomization checks 
may not be included in the trial monitoring report 
beyond stating that the check was conducted with 
results held confidentially by the trial statistics team. 
In addition, the randomization numbers should be 
checked to ensure that they have been allocated in 
chronological order with any missing randomization 
number accounted for with appropriate explanation. 
The correct implementation of the randomization process 
at sites is crucial. Any site specific randomization prob- 
lems should therefore be summarized in the trial mon- 
itoring report with only a minimum number of errors 
required to trigger further action. 

Missing primary outcome data 

Missing primary outcome data can affect the reliability 
of trial results, particularly when there is a substantial 
amount of missing data or if reasons for missing data 
are related to treatment or outcome. Procedures to min- 
imise the occurrence of missing data should always be 
considered and incorporated into the trial design, and 
repeatedly highlighted to staff throughout the trial. To 
allow for the potential impact on power it is usual prac- 
tice to inflate the estimated sample size to allow for a 
certain percentage of missing data. This inflation factor 
may be a reasonable upper value to define an acceptabil- 
ity threshold for missing primary outcome data to guide 
decision making and appropriate further action during 
monitoring activities. The cumulative percentage of par- 
ticipants with missing primary outcome at each site can 
be plotted (Figure 3) at various time-points, which need 
not necessarily be equally spaced. This graphical approach 
can facilitate interpretation and help easily identify sites 



which cross the acceptability threshold (5% is used in 
Figure 3), or which repeatedly remain at an unaccept- 
ably high level. The reason for missing primary outcome 
data should be identified and summarized along with 
the graph. Any missing primary outcome data should be 
routinely queried with the site as soon as possible. If the 
cumulative percentage of participants with missing pri- 
mary outcome data crosses the acceptability threshold 
the site should be contacted to discuss and reiterate the 
importance of complete data for this outcome. A subse- 
quent site visit should be arranged if the problem is stiU 
apparent in the next trial monitoring report. 

Patient safety 

The pharmacovigilance plan is specified in the protocol 
and considered appropriate to the risks identified in the 
trial and what is known about the interventions and 
conditions under study [13,14]. The IDSMC is respon- 
sible for monitoring patient safety throughout the trial 
and this would usually be done through a regular review 
of accumulating data. The trial monitoring plan would 
usually include details of safety reporting indicators 
(identified from the risk assessment) that are compared 
across sites to identify whether events are being identified 
and reported consistently, and in a timely manner. Ex- 
ample graphical approaches for comparing safety report- 
ing indicators are provided in Figure 4 (hypothetical data 
used for illustrative purposes only) where the key data 
item of interest is serious adverse events (SAEs). 

In Figure 4a, the cumulative percentage of participants 
with at least one event across the trial as a whole, and at 
each site, are plotted together against some measure of 
time (month in this example) which need not be equally 
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spaced. This can be used to check for under or over 
reporting of SAEs at sites, and is a useful approach if a 
small number of events of this particular type (SAEs in 
this example) are expected, and most likely to occur in- 
frequently per participant. Large discrepancies can be 
discussed and monitored closely over time to identify 
whether any patterns persist which may trigger further 
action. 

The approach in Figure 4a does not account for the 
length of follow-up of each individual participant. If 
there is variability in participants' length of follow-up 
and follow-up time is expected to increase the likelihood 
of an event such as an SAE, the approach summarized 
in Figure 4b may be preferable. Here, the overall event 
rate for the trial (sum of all events divided by the sum of 
all follow-up for the trial) is calculated, along with the 
expected number of events per site (sum of all follow up 
at a site x overall event rate for the trial). The difference 
between the actual and expected number of events at 
each site is plotted against time (Figure 4b). 



If the event of interest is expected to occur frequently 
with multiple events per participant, such as SAEs in a 
cancer trial, the plot in Figure 4c may be most appropri- 
ate. Here the overall mean number of SAEs per partici- 
pant per unit time (such as month) is calculated across 
all sites (sum of all SAEs divided by the sum of all 
follow-up) with a reference range (for example a 95% 
reference range = mean ±1.96 x sd, where sd = Vmean). 
It may be necessary to use a narrower reference range 
for early monitoring, or perhaps plot a 95% and 90% ref- 
erence range on the same graph. The number of SAEs 
per unit time for each individual participant (sum of all 
SAEs for participant j divided by the sum of all follow- 
up for participant j) is plotted along with the overall 
mean number and reference range. This may be pre- 
sented on a single plot highlighting outliers (with differ- 
ent symbol for each site) above or below the reference 
range or as site specific graphs comparing the distribu- 
tion of events. The site specific percentage of partici- 
pants with SAE rate outside the reference range can also 




Figure 4 Example graphical approaches (hypothetical data) for comparing serious adverse events (SAEs). (a) percentage participants 
witli at least one SAE, (b) discrepancy in SAE event, (c) SAE rate at site 1. 
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be calculated (or plotted over time) to guide interpret- 
ation. The trial monitoring plan should specify guidelines 
to follow to determine appropriate action if the SAE rate 
crosses a particular threshold (such as upper and lower 
limits of the reference range). 

Protocol deviations 

Key protocol deviations that might occur during the trial 
should be identified by the TMG (in conjunction with 
the risk assessment) and summarized in the trial monitoring 
plan. The TORPEDO-CF monitoring plan (Additional file 2) 
provides an example of a protocol deviation table, listing 
examples of potential deviations of important protocol 
specifications such as eligibility criteria, treatment regi- 
mens and study assessments. For each item, the potential 
protocol deviations that may occur are listed and graded 
in terms of the impact such a deviation would have on pa- 
tient safety and the reliability of trial results. This is gener- 
ally graded as major (potentially major impact either 
through introduction of bias to study results or effect on 
safety) or minor (potentially minor impact on patient and 
not likely to introduce bias), but the grade may vary ac- 
cording to the degree of deviation observed. Justification 
is also provided for the assessment of the impact of each 
potential protocol deviation. All protocol deviations that 
occur during the trial should be summarized within the 
trial monitoring reports with particular focus on major de- 
viations. If a particular deviation occurs frequently this 
may indicate the need for a protocol amendment. The 
graphical approaches described in Figure 3 and Figure 4 
may be appropriate for the comparison of protocol devi- 
ation rate to identify sites which have more frequent devi- 
ations from protocol thereby requiring further training or 
on-site visits. 

Case report form (CRF) completion monitoring 

Paper CRFs are routinely checked (manually or electronic- 
ally) when received to ensure that they are signed by 
authorised personnel. An electronic CRF tracking form 
keeps a record of CRFs received into the unit. Full details 
of the CRF tracking system should be included in the data 
management plan. A summary of data from the CRF 
tracking system, in particular missing CRFs, delay submit- 
ting CRFs (paper or electronic) to CTRC, completeness of 
reporting, and delay responding to queries can be gener- 
ated by site or other alternative as appropriate. If an indi- 
vidual site consistently fails to keep up to date with CRF 
(paper or electronic) completion and timely submission to 
CTRC, the TC should work with the site to develop a plan 
to correct this issue. 

Validation of data 

Range and consistency checks can be used to identify 
unUkely or implausible data and would usually be 



programmed into the trial database of all CTRC trials dur- 
ing database development. All validation and consistency 
checks are recorded in the trial data management plan 
and referenced within the trial monitoring plan. External 
sources of information may also be available to check the 
validity of certain data variables. For example, date and 
cause of death can be verified by using death registry data 
such as that recorded by the Office of National Statistics 
in the UK. 

Visit dates, or other important dates can be routinely 
examined centrally to check for consistency and accur- 
acy. For example, in some trials a weekend clinic visit 
would not be expected to occur and may indicate a data 
entry error or in some rare cases could suggest fraud. 
Visit date checks can also be used to identify visits 
which have occurred outside of protocol timelines or 
identif)^ visits that have been missed. In larger trials it 
may be preferable to present site specific scatter plots of 
actual date of visit compared to expected date of visit 
and compare the percentage of visits outside the permit- 
ted visit window across sites. Finally, it is possible to 
identif)^ inconsistent dates by cross checking different in- 
formation within, and across CRFs for a patient. 

Summarising the results of range, consistency and valid- 
ation checks across sites can provide useful information 
about compliance with the trial protocol and procedures, 
indicate resource issues at sites, and can be helpful sup- 
porting information when deciding whether site visits may 
be required. 

Pharmacy compliance with IMP handling procedures 
(clinical trials of IMPs only) 

Checks should be performed of the suitability of the 
environmental conditions (such as a central review of 
temperature logs), suitability of the product (expiry dates, 
potential degradation of the presentation) and checks 
that all medication can be accounted for via dispensing 
logs, checking that any medication returned by a partici- 
pant matches the amount dispensed minus the amount 
consumed. 

Quality assurance of statistical analyses 

All interim and final statistical analyses are undertaken 
in accordance with the relevant standard operating proce- 
dures and statistical analysis plan. The derivation of the 
primary outcome, corresponding statistical analyses, and 
safety data are independently programmed by a CTRC 
statistician independent to the trial. These independent 
analyses are cross checked with those undertaken by the 
trial statistician. This is particularly important in trials 
which include a primary outcome with a complex defin- 
ition that depends on multiple variables, requiring sub- 
stantial programming. 
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CTRC internal performance monitoring 

In addition to monitoring the performance of sites in- 
volved in the trial, it can often be important to monitor 
performance of the coordinating trials unit. One ex- 
ample would be to review the data entry process by 
comparing data entered by trials unit staff on the data- 
base against the data recorded on the paper CRF. This 
would usually be undertaken for all randomization re- 
lated data and primary outcome data as a minimum with 
additional key data items identified as important from 
the risk assessment. These reviews are useful to (a) as- 
sure data accuracy, and (b) to identify whether data 
entry staff at the CTRC require further training or sup- 
port. It may also be relevant to review other measures 
such as the time between receipt of paper CRF and com- 
pletion of data entry, or number and type of queries 
raised, to allow potential delays in the system to be 
highlighted so that appropriate action can be taken as 
early as possible, minimising the negative impact on accu- 
mulating trial data. A summary of these internal perform- 
ance reviews should be included in the trial monitoring 
report at the relevant time point, together with any re- 
quired action plan. 

Other issues 

Other important hazards that require central monitoring 
should be identified from the risk assessment and appro- 
priate procedures listed in the monitoring plan. For ex- 
ample, trials which rely on a blood test to guide treatment 
dosing would benefit from monitoring the number of in- 
valid or missing blood samples across sites; trials using 
variable dosing of treatments may summarize dose (for 
example median, interquartile range, minimum and max- 
imum) across sites which could identif)^ inappropriate 
dosing that may have an impact on patient safety; in 
blinded trials the emergency unblinding rate should be 
compared across sites with appropriate measures in place 
to assure that the correct procedures have been followed. 

On-site monitoring 

On-site monitoring involves a member of the trial man- 
agement team visiting an individual site to undertake 
monitoring activities. On-site monitoring may either be 
routine' in accordance with the monitoring plan, or may 
be 'triggered' as a result of the TMG review of trial moni- 
toring reports. The rationale, timing, frequency and activ- 
ities to be undertaken during routine on-site monitoring 
visits should be decided by the TMG in conjunction with 
the risk assessment, with appropriate consideration given 
to the rate of participant recruitment at each site, the 
complexity of trial procedures, and experience of the site 
in conducting clinical trials. 

If routine on-site monitoring is planned, the first site 
visit should ideally occur during the early phase of the 



trial following recruitment of the first few participants at 
the site to allow the monitor to pick up early issues, con- 
firm accurate interpretation of the protocol and data 
management procedures, and address any unresolved 
queries or problems in a timely manner. Site visits will 
normally involve the Principal Investigator (PI) and site 
research nurse/practitioner but may involve any or all of 
the individuals involved with the trial at the particular 
site. This may include for example a research radiog- 
rapher, trial pharmacist, or data manager at site. 

During routine site visits, focus would tend to be on 
making sure that trial procedures have been conducted in 
accordance with the trial protocol and according to the 
principles of GCP under the UK regulations. Any issues 
identified are discussed with site staff and explanations or 
remedial action documented within the site monitoring 
visit report which is sent to the site and also included in 
the TMG trial monitoring report. Routine on-site visits 
provide an opportunity for trial management staff to offer 
mentorship and additional training for site staff to ensure 
that procedures are being followed as outlined within the 
protocol. 

If the TMG review of trial monitoring reports identifies 
issues that cannot be resolved through contact with sites 
by email or telephone, or if significant issues are identified, 
a triggered site visit may be indicated. Examples of triggers 
might include: identification of clear differences between 
expected and actual recruitment rates, no recruitment for 
an extended period at site, identification of outlying 
screening failure rate at a particular site; missing primary 
outcome data above a pre-defined acceptability threshold; 
a higher or lower SAE reporting level at site in compari- 
son with the trial as a whole that cannot be reasonably ex- 
plained by knowledge of patient characteristics; repeated 
violations in the timing of consent and trial related proce- 
dures, repeated non-receipt of consent forms within the 
timelines specified in the protocol or inaccuracies in the 
completion of consent documentation; repeated use of su- 
perseded versions of the Patient Information Sheet and 
Consent forms (PISC); repeated evidence of protocol devi- 
ations (such as recruitment of ineligible participants); and 
repeated discrepancies in the assignment of randomization 
number at a given site. 

Following the triggered visit a site monitoring visit re- 
port will be completed detailing the reason for the visit, 
the processes reviewed, issues identified and discussed 
with the trial staff at the visit, as well as corrective mea- 
sures to be implemented. The site monitoring visit re- 
port should be reviewed and signed by the monitor and 
PI on site and should be included in the trial monitoring 
reports for review by the TMG. A follow-up visit will be 
arranged with the site staff to ensure that any action 
points outlined in the site monitoring visit report have 
been acknowledged and rectified. 
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The trial monitoring report 

The TMG are responsible for reviewing monitoring activ- 
ities undertaken during the trial and for authorising the 
appropriate subsequent action to be undertaken. These 
activities are summarized in a trial monitoring report 
which is prepared by the Trial Coordinator (TC) with in- 
put from other team members as appropriate, and is 
reviewed according to a regular schedule which should be 
pre-specified in conjunction with the risk assessment and 
detailed in the monitoring plan. Higher risk trials, or 
fast recruiting trials, may require more frequent reviews 
of trial monitoring reports, and it may not always be ne- 
cessary to present all monitoring activities in every trial 
monitoring report. For example, a trial experiencing re- 
cruitment difficulties may require additional TMG review 
of recruitment patterns and screening failure rates. The trial 
monitoring report discussed at these meetings would not 
necessarily require all pre-planned monitoring activities to 
be summarized. The frequency of review of trial monitoring 
reports may also need to be amended during the trial if is- 
sues that require immediate action are identified. 

The content of the trial monitoring reports will vary 
by trial but would generally include summary results of 
the central monitoring activities, statistical monitoring, 
issues raised from TSC and open IDSMC meetings, and 
on-site monitoring, with any relevant site monitoring 
visit reports included as an appendix to the report. 

To ensure that there is a record of any decisions made 
by the TMG, the trial monitoring report includes an 'ac- 
tions' section listing all actions requested as a result of 
TMG review. This list of actions would also be included 
in the subsequent trial monitoring report for review by 
the TMG to ensure that all previous actions have been 
carried out with a satisfactory outcome. 

A site may be closed on the authority of the sponsor if 
they have not enrolled any participants for a consider- 
able amount of time or the enrolment rate is not accept- 
able, or if the site is non-compliant with trial procedures 
or regulatory requirements. During review of the trial 
monitoring report, the TMG will consider whether any 
serious breaches in Good Clinical Practice have occurred 
which would be reported to the Sponsor who would be 
responsible for informing the MHRA. 

Although there may be some overlap in content, this trial 
monitoring report should not be confused with reports 
which are prepared by the trial statistician and presented to 
the IDSMC. The IDSMC reports are usually for review 
only by the IDSMC since they often include results of ana- 
lyses that compare treatments and these interim results 
could potentially introduce bias if released to the trial team. 

Conclusions 

Clinical trials require measures to be taken to assure the 
quality of data, reliability of results, and to protect 



participants' rights and safety. Recent developments in the 
literature by international bodies and regulatory agencies 
[3-6] have supported the need for risk-proportionate ap- 
proaches to monitoring. The risk-proportionate approach 
adopted by the CTRC, an active, non-commercial UK 
clinical trials unit, is described in this paper to aid dissem- 
ination of methods, promote discussion and contribute to 
the evidence base that is currently lacking. However, fur- 
ther empirical evidence is required to thoroughly evaluate 
the costs, and advantages and disadvantages of alternative 
methods. Further, more in-depth statistical monitoring 
may be required for the detection of fraud [12], or to sup- 
plement simple approaches [2] if problems are highlighted. 
The use of statistical monitoring methods is an area of ac- 
tive research and appropriate use of these methods requires 
careful consideration of issues such as multiple testing and 
trial size. 

Recommendations made by the CTTI project on effect- 
ive and efficient monitoring as a component of quality 
highlight that no single monitoring approach is appropri- 
ate or necessary in all circumstances', and that the moni- 
toring approach for a given clinical trial should be tailored 
to the needs of that trial and may combine several 
methods' [3]. These points, along with other recommen- 
dations made by CTTI (many of which are reflected in the 
CTRC approach) should be kept in mind when developing 
the trial monitoring plan in conjunction with the trial risk 
assessment. Furthermore, by publishing the approach 
taken by CTRC we are supporting an ancillary recommen- 
dation made by the CTTI project, to 'Share knowledge 
and experiences' [3], and we fully encourage further shar- 
ing and discussion amongst the trials community so that 
best practices may be established. 
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